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ABSTRACT 

A study  of  coding  to  reduce  the  frequency  of  errors  in 
communication  systems  which  transmit  data  at  a con- 
stant rate  has  been  started.  A new  single-error-cor- 
recting  code  (the  Wagner  code)  is  described  and  ana- 
lyzed. Its  performance  in  a constant-data-rate  system 
is  evaluated  and  compared  with  Hamming's  single-error- 
correcting  code.  The  Wagner  code  is  superior  for  many 
communication  applications.  Its  successful  implemen- 
tation does  not  require  too  precise  equipment. 
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CODING  FOR  CONSTaNT-DATA-RATE  SYSTEMS 

PART  1.  A NEW  ERROR-CORRECTING  CODE 


A.  INTRODUCTION 

In  this  report,  we  shall  consider  a communication  system  in  which  data  consisting  of 
sequences  (known  as  words)  of  binary  digits  are  transmitted  at  a predetermined  constant  rate. 

(For  our  purposes,  a binary  digit  is  one  of  two  electrical  signals  of  duration  T and  bandwidth  W.  ) 
The  nature  of  the  data  and  the  manner  in  which  they  are  translated  into  words  are  irrelevant  to 
this  discussion.  For  example,  the  data  may  be  English  letters,  numbers,  etc.,  reduced  to  se- 
quences of  five  binary  digits  each  for  use  in  a teletype  system,  or  they  may  be  conventional  sym- 
bols representing  entire  messages. 

A basic  problem  of  coding  is  to  reduce  the  average  rate  of  incorrectly  received  words 
as  much  as  possible.  Accordingly,  additional  digits  are  added  to  the  word  for  the  purposes  of 
error  detection  or  correction.  The  assumption  that  the  words  are  sent  at  a constant  rate  requires 
that  each  binary  digit  be  shortened  by  such  an  amount  that  the  coded  words  (message  digits  plus 
check  digits)  have  the  same  duration  as  the  original  uncoded  words.  This  shortening  of  each  digit 
increases  its  probability  of  error  and,  consequently,  the  probability  of  error  per  word.  On  the 
other  hand,  the  coding  imposes  constraints  on  the  digits  composing  a word,  so  that  errors  may 
show  up  as  inconsistencies  and  may  in  many  cases  be  corrected.  This  tends  to  reduce  the  prob- 
ability of  error  per  word.  The  efficacy  of  a code  depends  on  how  much  the  second  effect  out- 
weighs the  first. 

The  simplest  code  of  all  consists  of  an  extra  digit  selected  to  make  the  sum  of  all  the 
digits  in  the  coded  word  even  (or  odd).  If  the  sum  of  the  digits  of  the  received  word  has  the  wrong 
parity,  an  odd  number  of  errors  is  known  to  have  been  made.  There  is,  however,  no  indication 
of  the  correct  replacement  for  the  mistaken  word,  unless  some  dependence  between  separate 
words  (such  as  the  redundancy  of  printed  English1)  is  exploited. 

Hamming2  has  devised  a code  that  corrects  all  single  errors-  It  consists  of  adding 

k suitably  chosen  check  digits  to  the  m message  digits.  if  another  digit  is  added,  double  errors 

? 

can  be  detected  as  well  as  single  errors  corrected.-  In  this  paper,  we  describe  a new  singlc- 
error-correcting  code  (the  Wagner  cede)  and  evaluate  its  performance  in  a constant -data -rate 
system,  particularly  as  compared  with  i'nai  of  the  Hamming  code. 

3 4 

Recently,  multiple -error-correcting  codes  have  been  constructed.  ’ An  analysis  of 
their  performance  in  constant -data -ci  te  systems  will  be  published  in  Technical  Report  No.  48, 
"Coding  for  Constant  - Data  - Rate  Systems,  Part  II  - Multiple-Error-Correcting  Codes." 

B.  DESCRIPTION  OF  THE  WAGNER  CODE 

In  this  study,  we  are  concerned  with  communication  systems  that  transmit  words  con- 
sisting of  binary  digits.  A binary  dig-t  is  one  of  two  electrical  signals  Xj(t)  and  x.,(t)  of  duration 
T and  bandwidth  W.  Let  p(x./y)  be  '.he  (a  posteriori)  probability  that  if  y is  received,  was 

♦These  two  requirements  ore,  strictly  speokinr,  incompotible  (see  Appendix). 

*♦  For  m - 2 to  4,  k - 3;  for  m - 5 to  1 1,  k - 4;  etc.^ 
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tent,  and  let  Ap  be  p(Xj/y)  - p(x2/y).  In  the  absence  of  any  constraints  on  the  digits  composing 
a word  or  of  dependence  between  the  words  themselves,  the  receiver  can  compute  only  p(x(/y) 
and  p(x.,/y),  and  for  each  digit  choose  Xj  or  x^,  depending  on  whether  p(Xj/y)  or  p(x.,/y)  is  the 
larger  (or.  equivalently,  whether  Ap  is  positive  or  negative).  The  error -correcting  code  that 
we  shall  describe  (named  the  Wagner  code  after  C.  A.  Wagner  of  this  laboratory,  who  suggested 
the  basic  idea)  enables  us  to  use  some  of  the  information  presented  by  the  magnitudes  of  the 
Ap's*  by  intioducing  a constraint  on  the  digits  composing  a word.  This  information  is  ignored 
by  more  conventional  codes. 

In  the  Wagner  code,  a transmitted  word  consists  of  a sequence  of  m message  digits 
and  an  additional  digit  used  as  a parity  check.  As  each  of  the  perturbed  digits  y arrives  at  the 
receiver,  the  a posteriori  probabilities  p(Xj/yi  and  pfx^/y)  are  calculated.  Each  digit  of  the  re- 
ceived sequence  is  tentatively  identified  as  Xj  or  x^,  depending  on  whether  p(x(/y)  or  pfx^/y)  is 
the  larger,  and  the  values  of  the  a posteriori  probabilities  are  stored  in  a memory  for  the  dura- 
tion of  a word.  The  sequence  thus  obtained  is  checked  for  parity.  If  the  parity  is  correct,  the 
word  is  printed  as  received.  If  the  parity  check  fails,  the  digit  for  which  the  difference  Ap  be- 
tween a posteriori  probabilities  is  the  smallest  is  considered  the  digit  most  in  doubt,  and  the 
word  is  printed  with  this  digit  altered.  The  receiver  then  clears  the  stored  values  of  the  proba- 
bility differences  from  the  memory  and  proceeds  to  the  next  word. 

Thus  we  may  characterize  the  Wagner  c ode  as  one  which  probably  corrects  single 
errors.  (Multiple  errors  are  always  printed  incorrectly.)  However,  as  we  shall  see,  it  ran  be 
more  effective  in  a constant-data -rate  system  than  a code  that  corrects  all  single  errors  (such 
as  the  Hamming  code). 

The  a posteriori  probabilities  p(Xj/y)  and  p(x2/y)  are  functions  of  the  random  received 
waveform  y (t ) and  therefore  are  themselves  random  variahles.  The  calculation  of  their  distribu- 
tions is,  in  general,  very  difficult.  For  simplicity,  wc  shall  consider  the  case  where  the  two 
transmitted  signals  have  equal  er.cigy  and  equal  a priori  probabilities  and  are  perturbed  by  the 
addition  of  white  Gaussian  noise.  It  has  been  shown1  that  for  this  case 


and 


p(x,/y)  --  p exp 


U)  y(t)  dt 


P(x2/y)  = P exp 


x2(t)  y(t)  dt 


(1) 


where  p and  y are  constants.  Thus  the  transmitted  signal  with  the  larger  correlation  has  the 
larger  a posteriori  probability.  Equivalently,  we  may  write 


where 


P(X[/y) 

= exp  Itfzj  -z2)]  . 


x ,(t)  y(t)  dt 


and 


x2(t)  y(t)  dt 


(2) 

(3) 


* 1 he  i-gne  cf  *he  ore  us*-1  in  making  the  tentative  identification  of  the  transmitted  word. 
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From  Eq.(2),  we  see  that  the  smaller  >he  difference  az  = z^  - z^  between  the  correlation  inte- 
grals. the  closer  to  unity  the  ratio  of  n posteriori  probabilities  and,  consequently,  the  smullet 
the  difference  Ap  between  the  two  probabilities.  Thus,  if  the  parity  check  fails,  the  digit  that 
should  be  changed  (as  the  one  most  in  doubt)  is  the  one  for  which  Az  is  the  smallest. 

It  is  shown  in  the  Appendix  that  z^  and  z^  are  normally  distributed  random  variables 
(with  means  Cj  and  c^,  and  variances  o-j  and  )*  so  that  calculations  are  especially  simple  for 
the  correlation  detector*  Moreover,  under  the  assumptions  made  above,  the  correlation  detec- 
tor is  equivalent  to  the  probability  detector.  Therefore,  it  is  assumed  in  what  follows  that  de- 
tection is  by  correlation. 


ANALYSIS  OF  THE  WAGNER  CODE 


1.  Probability  of  Error  Per  Digit 

As  noted  above,  the  correlation  integrals  z^  and  z^  (corresponding  to  the  signal  that 
was  sent  and  the  signal  that  was  not  sent,  respectively)  are  random  variables  with  probability 
densities  , 

1 


and 


W(Zj)  = 


n/Ttt  ( 


exp 


W<*2>= 

*/Tir  c .j 


exp 


r ( vlLlLI 

L -1  ' 


r (Zz~czh 

Z«}  J 


(4) 


15) 


It  is  shown  in  the  Appendix  that  if  Xj  and  x2  are  suitably  chosen,  then  Zj  and  z2  may  be  regarded 
as  statistically  independent  random  variables.  Accordingly,  the  probability  density  of  finding  a 
separation  Az  = Zj  - z2  is 


W(Az  ) = exp 

si  2na 


(6) 


2 2 2 

where  Ac  = Cj  - c2>  0 and  <r  - . 


Here  we  make  use  of  a well-known  theorem  on  the  dis- 

b 


tribution  of  the  sum  of  independent,  normally  distributed  random  variables.  If  Az  is  negative, 
then  selecting  as  the  transmitted  signal  the  signal  giving  the  larger  correlation  integral  will  re- 
sult in  an  error.  Thus  the  probability  of  error  per  digit  is 


p(a)  = r W(Az)  dAz  = j ( 1 - erf  a)  , 

„ J -00 


(7) 


where  a = Ac /\T2a.  The  parameter  a,  which  is  proportional  to  the  signal -to-noise  ratio  of  the 


*For  simplicity  of  notation,  we  shall  always  use  the  subscript  "one’  for  the  signol  thot  was  transmitted.  Thus 
Cj  > Cj,  and  Z)  < zj  results  in  an  error. 

**The  existence  af  the  simple  expression  (7)  far  p(a)  was  pointed  out  by  Dr.  I,  S.  Reed,  who  evaluated  the  equiva- 
lent form 

p,“’  ■ 1 ,xp  [-  !1SL]  f.l ex p [-  di  ds  - 


by  a rotation  af  45  in  the  (s,t)  plane. 
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correlator  difference,  is  the  significant  parameter  in  the  calculations  that  follow. 

Since  we  do  not  know  which  digit  was  actually  sent,  we  do  not  know  the  sign  of  Az. 
From  Eq.(6),  the  joint  probability  that  the  correlator  difference  lies  between  |az|  and  | Az  | + 
d|Az|  , and  that  the  larger  correlation  integral  corresponds  to  the  transmitted  signal,  is  d|Az 
times 

(|Azl  - Ac. f 


1 


W(  | Az  1, right ) = exp 

S / 2 it  or 


la 


(8) 


On  the  other  hand,  the  joint  probability  that  the  correlator  difference  lies  between  | Az  [ and 
|Az|  + d|Az|,  and  that  the  larger  correlation  integral  does  not  correspond  to  the  transmitted 
signal.  isd|Az|  times 

1 ..  f ( | Az  | + A cf 


W(  | Az  |,  wrong) 


exp 


\l  2ir  <r  L 

2.  Probability  of  Correcting  a Single  Error 


l<r 


(9) 


Suppose  that  the  received  word  has  n digits.  Since  the  parity  check  fails  if  a single 

error  is  made,  and  sinc  e then  the  Wagner  code  changes  the  digit  with  the  smallest  | Az  | , the 

probability  Iln(u)  that  a single  error  is  made  and  is  i orrected  by  the  Wagner  code  is  just  the 

probability  that  the  digit  with  the  smallest  |Az|  is  incorrect  and  that  the  n - 1 other  digits  are 

correct,  [I  (a)  can  be  calculated  as  follows, 
n'  ' 

Let  |Azi|  be  the  correlator  difference  for  the  i-th  digit.  Since  the  Iaz^  are  inde- 
pendent random  variables,  the  joint  probability  that  the  first  digit,  with  correlator  difference 
between  | AZj  | and  jAzJ  + d|AZj|,  is  wrong,  and  that  all  the  other  digits,  with  correlator  differ- 


ences between  lAz^l  and  |Az2| 


+ d | Az  - 1 | Az  | and  | Az 


+ d|Azn|,  are  right,  is 


n n 

W(|Az|, wrong)  f]  W(  | Az  |.  right ) n d|Az.| 
i=2  1 i=l  1 


(10) 


Thus  the  joint  probability  that  the  first  digit  is  wrong,  that  the  n - 1 other  digits  are  correct, 
and  that  | Az^  | < | <A.z  ^ } < . . . < | Az^  | is 


J*oo  /»  I az  | 

W(|Az  |, right)  d|Az  | I n W(|Az  |,  right)  d | Az  | 

o Jo 

r I az,|  r |az  | 

...  W(|  Az2|, right)  d|  Az2|  j W(]  AZj  |,  wrong)  d|AZj  | 

Jo  Jo 


(11) 


Since  there  are  in  all  nl  orderings  of  the  n correlator  differences,  the  joint  probability  Il^a) 
that  a single  error  is  made  in  a word  of  n digits  and  that  it  is  corrected  by  the  Wagner  code  is 
fiiven  by 

n"(“' = Jo  75F7  exp  [- 17  l|Ai"'  - ACM  d|“»' 


/.  ■ 


n/Ttt  ( 


exp 


[-17 


Az 


n-  1 


- Acf  d|  Az 


n-1 1 
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3 1 <?xp L (|  AzJ  - Ac)2 

2<T 


nTTtt  (T 

|Az2l  , 


d|  Az 


exp L ( | AZj  | + Acf  I d|  -iZj  | 

sj  2. tt  tr  L Zcr 


-r 

x 

f exp[-(xn-af]  dxn  P nexp[-(x  -a)2]dx 
( n/t?)  ^ o Jo 

. ..  I 3 exp  [-  (x^  - a)2]  dx2  j 2 exp  [—  (Xj  + af]  dXj 
J O J o 


(12) 


where  a = Ac/\TZ<r  as  in  Eq.(7).  The  c conditional  probability  that  a single  error  will  be  corrected 
by  the  Wagner  code,  assuming  that  a single  error  has  been  made  in  an  n-digit  word,  is  given  by 


nn(a> 

nqn‘  '(a)P(a) 

where  p(a)  is  the  probability  of  error  per  digit 
p(a)  = ^ (1  - erf  a) 

and 

q(a)  = 1 - p(a)  = j (1  + erf  a) 

Equation  (13)  follows  immediately  from  the  definition  of  conditional  probability  and  the  fact  that 
nqn_1(a)p(a)  is  the  probability  of  a single  error  in  an  n-digit  word. 

It  is  desirable  to  have  ^n(a)  in  a form  more  suitable  for  numerical  computations. 
Thus  we  reduce  the  n-fold  multiple  integral  (12)  to  a convenient  recurrence  formula.  Let 


(13) 

(?) 

(14) 


F (a)  = 

n ' ' 


. (v~) 


n'  ' - n:  n 


n.(a)  ■ 


and 


fn(x.a)  = f dxn  exp  [-  (xn  - of)  f n dxnl  exp  [- (xn  j - of  j 
J o Jo 

...  f 3 exp  [- (x2  - a)3]  dx2  f 2 exp  [- (Xj  + af ] dx 
Jo  Jo 


From  Eqs.(  15)  and  (16)  it  follows  that 


f (oo,a)  = [I  (a);  f (0,a)  = 0 
n'  ' n'  ' n'  ' 


and 


Since 


Fn(Q)  -f  dxnexP[-(xn-a^]  fn_i(xn.a)  . 
J o 


37  fn(x,a)  = exp  [-  (x  - a)2]  fn_  j(x,a) 


(13) 


! 16) 


(17) 

(18) 

(18) 
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and 


exp  [—  (x  - a) 


n/  tt  d 
Z dx 


erf  (x  - a) 


(20) 


we  can  reduce  Eq.(18)  by  repeated  integration  by  parts  to 

Fn<°>  = ^ Fn-l(Q)-^)2  I Fn-2<Q)  +^)3  V.  Fn-3<Q>  + " 


* f — n-2  , 

.n*l  ,v  w , 1 


+ (-»)  (V) 


+ (--i)n  (#) 


Z I (n  - 2):  ‘2 
n - 1 


F,(a) 


~-T):  p(a)  ~ J"  epfn"  1 (X  - Q)  exp  [- (x  + Q)2]  dx 


Finally,  by  the  definition  [Eq.(lS)], 

n„<»>  ■ * ‘t&Q 

+ (p^rQ,p(o)-T1n(Q)l  • 

I (a)  - f erfn  1 (x  - a)  exp  (-  (x  + a)2]  dx 

n vHT  Jo 


(Q) 


where 


(21) 


} 


(22) 


(23) 


ln(a)  and  fJn(a)  have  been  computed  for  values  of  n from  2 to  9;  the  results  are  tabulated  in 
Tables  I and  II  for  selected  values  of  a. 


I 

! 

i 


I 

i 

i 

j 


s 


TABLE  I TABLE  II 


Values  of  In(a)  [ 

see  Eq.  (23)] 

Values  of  0 n(Q) • the  joint  probability  that 
a single  error  is  made  in  an  n-digit  word 
and  that  it  is  corrected  by  the  Wagner  code 
[see  Eq.  (22)]. 

1.0 

1.5 

2.0 

3.0 

1.0 

1.5 

2.0 

3.0 

2 

-0.09943 

-0.03062 

-0.004604 

-0.00002210 

2 

0.128 

0.0323 

0.00464 

0.0000221 

3 

0.07161 

0.02797 

0.004533 

0.00002210 

3 

0.160 

0.0462 

0.00691 

0.0000331 

4 

-0.05208 

-0.02571 

-0.004466 

-0.00002210 

4 

0.181 

0.0588 

0.00914 

0.0000442 

5 

0.03932 

0.02375 

0.004401 

0.00002210 

5 

0.193 

0.0704 

0.01134 

0.0000552 

6 

-0.02979 

-0.02201 

-0.004339 

-0.00002209 

6 

0.199 

0.0811 

0.01350 

0.0000663 

7 

0.02304 

0.02046 

0.004279 

0.00002209 

7 

0.200 

0.0908 

0.01564 

0.0000773 

8 

-0.01782 

-0.01906 

-0.004221 

-0.00002209 

8 

0.199 

0.0997 

0.01774 

0.0000884 

9 

0.01400 

0.01779 

0.004165 

0.00002209 

9 

0.195 

0.1079 

0.01982 

0.0000994 
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It  can  be  shown  from  the  original  integral  forms  [Eqs.(23)  and  (12)]  that 


and 


I (0)  - - 

n'  ' n 


n„«»  ■ ? 


(24) 


(25) 


Equation  (25)  may  also  be  derived  by  the  following  argument.  Since  a = 0 corresponds  to  a mean 
value  of  zero  for  the  correlator  differences,  any  digit  is  equally  likely  to  be  right  or  wrong.  The 
probability  of  one  incorrect  digit  and  n - 1 correct  ones  is  thus  n/2n,  and  the  probability  that  the 
incorrect  digit  has  the  smallest  correlator  difference  is  just  l/n  of  this,  giving  Eq.(25). 

For  large  a,  we  obtain  the  asymptotic 

forms 

(26) 
(27) 


I (a) 
n'  ' 


(-1)"-1  (i  _ erf  a) 


and 


nn(°) 


1 - erf 


np  (a) 


Equation  (27)  is  the  form  to  be  expected.  For 
large  a,  q(a)  ~ 1,  so  that  np  (a)  q11"1^),  the  prob- 
ability of  a single  error  in  an  n-digit  word,  is  ap- 
proximately np  (a).  Moreover,  as  we  shall  see 
below,  for  large  a the  wrong  digit  is  almost  cer- 
tainly corrected  by  the  Wagner  code.  Therefore, 
for  large  o 


nn'Q> 


np  (a) 


(28) 


Fig.  1.  Conditional  probability  of  corriction 
for  Wagner-cadad  words  of  5 and  8 digits. 


From  inspection  of  Tables  I and  II,  it  is  seen  that 
the  asymptotic  forms  [Eqs.(26)  and  (27)]  are  excel- 
lent approximations  for  a •*.  3. 

The  conditional  probability  that  if  one 
error  is  made  it  will  be  corrected  by  the  Wagner  code  is  given  by  Eq.(13).  Curves  of  this  prob- 
ability as  a function  of  a for  words  of  5 and  8 digits  are  shown  in  Fig.  1.  The  conditional  proba- 
bility approaches  unity  for  large  a,  i.e.,  for  large  signal-to-noise  ratio  of  the  correlator  differ- 
ences, almost  all  errors  are  corrected.  Below  values  of  a of  about  1.5,  the  conditional  proba- 
bility drops  sharply,  becoming  l/n  for  a = 0 [see  Eq.(25)].  As  is  to  be  expected,  the  conditional 
probability  for  a given  a decreases  with  increasing  n. 


D.  PROBABILITY  OF  ERROR  FOR  WAGNER- CODED  WORDS  - COMPARISON  WITH 
UNCODED  AND  HAMMING- CODED  WORDS 

The  probability  of  error  per  word  for  a Wagner-coded  word  containing  m message 
digits  (n  = m + l digits  in  all)  is 

Pw=  1 -qm+1(o)_  nm  + ,(«)  . (29) 

that  is,  the  probability  of  error  is  one  minus  the  sum  of  the  probability  that  the  word  is  received 
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correctly  and  the  probability  that  a single  error  is  made  and  then  corrected.  We  wish  to  com- 
pare P^,  with  Py  the  probability  of  error  per  word  if  no  rode  is  used,  and  Pjj,  the  probability  of 
error  per  word  if  the  Hamming  single -error- < orrec t ing  code  is  used.  Since  we  are  concerned 
with  eonstant-data-rate  systems,  the  duration  of  the  transmitted  signals  must  be  altered  if  coded 
words  (message  digits  plus  error-correc  ting  digits)  are  to  have  the  same  duration  as  differently 
coded  or  uncoded  words  Changing  the  signal  duration  c hanges  the  variance  of  the  correlator  dif- 
ference and  consequently  the  value  of  the  parameter  a and  the  probability  of  error  per  digit  [see 

7 

Eq.(7)].  For  large  TW  (the  only  case  of  practical  interest),  it  can  be  shown  that  the  variance  of 
the  output  of  any  averaging  devic  e is  proportional  to  T‘L  Thus  o j,  a ^ and  ,r  = -./  tr ^ are  all 

proportional  to  T *,  so  that  a(  - &c/\TT<r)  is  proportional  to  Using  this  result,  we  find  that, 

for  the  same  value  of  a used  in  Eq.(29), 


and 


p ..  - 1 -q 


'L-sr1  ") 


H 


1 - q‘ 


l+k  ( I m + 1 \ , , . m+k-1  / / m * l \ I I m + 1 \ 

W rrPTk  a)  - (m  + k)  q W nTTT  Q)  p [-J  °) 


(30) 


(31) 


where  k is  the  number  of  check  digits  required  by  the  Hamming  code  (see  above).  The  three 
quantities  [Eqs.(29),  (30)  and  (31)]  have  been  computed  for  values  of  m from  4 to  8 and  for  se- 
lected values  of  a.  The  results  are  given  in  Table  111. 


TABLE  III 


Probabilities  of  error  per  word  for  uncoded,  Hamming-coded, 

and  Wagner-coded  words  containing  m message  digits 

m 

a 

pu 

PH 

pw 

4 

1.0 

0.209 

0.191 

0.143 

1.5 

0.0349 

0.0248 

0.0115 

2.0 

0.00313 

0.00145 

0.00030 

3.0 

42  X 10~7 

6 X 10-7 

< 10-7 

5 

1.0 

0.269 

0.310 

0.190 

1.5 

0.0493 

0.0513 

0.0164 

2.0 

0.00486 

0.00375 

0.00045 

3.0 

84  x 10-7 

25  x 10-7 

< 10-7 

6 

1.0 

0.325 

0.335 

0.236 

1.5 

0.0641 

0.0530 

0.0220 

2.0 

0.00673 

0.00  346 

0.00062 

3.0 

138  X 10-7 

17  X 10-7 

< 10-7 

7 

1.0 

0.377 

0.362 

0.282 

1.5 

0.0789 

0.0552 

0.0281 

2.0 

0.00871 

0.00330 

0.00082 

3.0 

201  x 10-7 

12  x io-7 

< 10-7 

8 

1.0 

0.425 

0.388 

0.326 

1.5 

0.0937 

0.0580 

0.0347 

2.0 

0.01075 

0.00322 

0.00103 

3.0 

272  X 1CT7 

9 x 10-7 

< 10-7 

*The  result  thot  o is  proportionol  to  T ^ requires  that  the  integrols  in  Eq.  (3)  be  normoiized  by  dividing  them  by  T. 
The  result  that  a is  proportionol  to  is  independent  of  the  normolizotion. 
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There  is  only  a certain  range  of  values  of  the  signal-to-noise  ratio  of  the  correlator 
differenc  e for  which  it  is  worth  the  effort  to  implement  either  of  the  error -cor recting  codes. 

For  high  signal-to-noise  ratio,  very  few  errors  are  made,  and  additional  equipment  is  generally 
not  justifiable.  On  the  other  hand,  for  low  signal-to-noise  ratio,  multiple  errors  become  too 
frequent,  and  single-error-correcting  codes  are  of  little  use.  Thus  single-error-correcting 
codes  are  of  considerable  value  for  values  of  a from  about  1.0  to  3.0. 

As  shown  in  Table  111,  in  this  range  of  values  the  probability  of  error  of  Wagner- 
coded  words  is  considerably  less  than  that  of  Hamming-coded  words.  For  increasing  word 
length,  the  advantage  of  the  Wagner  code  diminishes  from  two  causes:  (1)  the  ratio  k/m  de- 
creases with  increasing  m so  that  the  length  of  the  digits  in  the  Hamming- coded  word  approaches 
those  of  the  Wagner-coded  words,  thus  narrowing  the  gap  between  the  corresponding  signal-to- 
noise  ratios;  and  (2)  the  conditional  probability  of  correcting  a single  error  decreases  with  in- 
creasing m.  Nonetheless,  even  for  m = 8 and  a = 2,  we  may  expect  only  103  errors  per  100,000 
words  using  the  Wagner  code,  as  compared  with  322  errors  per  100,000  Hamming-coded  words 
and  1,075  errors  per  100,000  uncoded  words. 

E.  IDENTIFICATION  OF  THE  SMALLEST  CORRELATOR  DIFFERENCE 

Until  now  we  have  assumed  that  the  receiver  can  pick  out  the  smallest  correlator 
difference  with  infinite  precision.  Suppose,  however,  that  the  equipment  used  in  implementing 
the  Wagner  code  is  such  that  the  smaller  of  two  correlator  differences  within  iAc  of  each  other 
cannot  be  identified  with  certainty.  Let  ( be  the  revised  probability  of  error  per  word,  as- 
suming that  single  errors  remain  uncorrected  whenever  any  correlator  difference  lies  within 
t Ac  of  the  smallest.  In  this  notation,  the  Pw  used  above  is  P^  t 

Suppose  that  the  first  digit  has  the  smallest  correlator  difference  |AZj|  and  is  known 
to  be  the  only  one  in  error.  Then  the  probability  that  | AzJ  , the  correlator  difference  of  the  i-th 
digit  (i  # 1),  lies  within  < Ac  of  |A  z | is  given  by 

prob  ( | Az,  | < Az  , < | Az,  I + ( Ac/  Az,  > | Az,|  ) 

Az,  | +t  Ac 

= t W(Az.)  dAz.  (32) 

prob(Az.  > | AzJ  ) Azj| 

|a z.  | +<ac  /-«, 

1 W(Az)  dAz  / W(Az)  dAz 

AZ]1  | tj  ] Az,  I 


•The  receiver  may  also  have  difficulty  in  determining  the  sign  of  smott  correlator  differences.  The  percentage  in- 
creases in  Py,  and  Pyy  produced  by  this  difficulty  ore  af  the  order  of  magnitude  of 

<PW,r -PW,0)/PW,0  - 

and  hence,  as  we  shall  see,  very  small. 

t For  simplicity  of  calculation,  we  have  chosen  Pyy  os  the  probability  of  error  per  word,  ossuming  that  single  errors 
remain  uncorrected  whenever  two  or  more  correlator  differences  lie  within  <Ac  of  each  other.  In  practice,  o smaller 
probability  of  error  can  be  obtained  by  the  simple  expedient  of  arbitrarily  changing  the  first  digit  of  every  set  of  digits 
with  correlator  differences  within  c of  eoch  other.  In  this  woy,  we  correct  olmost  half  of  the  errors  which  remain  un- 
connected in  calculating  P^f. 
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where  W(Az)  is  given  by  Eq.(6).  [Since  the  correlator  difference  is  always  positive  for  a correci 
digit,  the  absolute  value  sign  is  not  needed  on  |AzJ.  The  factor  prob(Az^>  |AZj|)  in  the  denom- 
inator renormalizes  the  distribution  W(Az^)  from  one  in  which  all  values  of  Az^  are  possible  to 
one  which  only  allows  values  of  az.  > |AZj|.] 

The  probability  [Eq.(32)]  is  written  for  a given  | Az^  | . Let  J(«  ,a)  be  the  probability 
that  Az . will  lie  within  « Ac  of  | Az^  | , no  matter  what  the  value  of  | AZj  | . It  is  clear  that  J(<  ,o) 
is  independent  of  i.  Using  Eqs.(6)  and  (9),  we  find 


J(<  ,a)  =1  W(  | AZj  | , wrong)  prob  ( | Az^  | < Az . < | AZj  | + < Ac/Az^  > | AZj  | ) d | AZj  | 

Jo 


-JL- f 

*/Ttr  a -'o 


(|Az.  | + Ac) 


exp  - 


2a 


r 1 Az,  1 +t  Ac 

p 

I 1 I 1 

(Az  — Ac) 

j,  , exp 

Z 

^ | Azt  | 

2<r 

dAz 


d Az 


(Az  - Ac)2 

, , exp 

> 

' 1 AZj  | 

L 2<r  J 

dAz 


(33) 


1 r°°  r . , ,2,  erf(x  - a + i a)  - erf(x  - a)  , 

= jo  exp t- <x  + a> i --i-T-iH[x-ar-,J  dx  • 

The  probability  that  Az.  does  not  lie  within  « Ac  of  | Az^  | , assuming  that  Az^  > | Az^  | , is  thus 
1 - J(<  ,q),  and  the  probability  that  none  of  the  n - 1 correct  digits  has  a Az  within  c Ac  of  | AZj  | 
is  [ 1 - J(<  .q)]""1.  Thus  we  find 


pw.<  ■ 1 - qm+1(Q)  - U - J(«  .°)]m  nm+l(.) 


(34) 


Calculated  values  of  J(0.1,a)  for  selected  values  of  a are  given  in  Table  IV,  together  with  a com- 
parison of  Pw  Q j and  Pw  Q for  words  of  six  message  digits. 

From  Table  IV,  we  see  that  even  for 
a rather  crude  receiver,  which  cannot  distinguish 
correlator  differences  lying  within  0.1  Ac.  of  each 
other,  the  percentage  change  in  the  probability  of 
error  per  word  is  at  most  about  two  per  cent  in 
the  region  of  interest.  Thus  the  advantage  of  the 
Wagner  code  over  the  Hamming  code  does  not  de- 
pend on  great  precision  of  the  correlators  or  the 
memory. 


TABLE  TV 

Selected  values  of  J(0.I,a)  and  comparison  of 

P 

W.  0 

and  P,„  . for  words  of  6 message  digits 

W , U.l 

a 

J(0.I,Q) 

P 

W,0 

PW,0.1 

1.0 

0.00379 

0.236 

0.241 

1.5 

0.000426 

0.0220 

0.0222 

2.0 

0.0000189 

0.00062 

0.00062  4 

F.  SUMMARY  AND  CONCLUSIONS 

Following  a suggestion  of  C.  A.  Wagner,  we  have  constructed  a code  that,  unlike  those 
previously  considered,  makes  explicit  use  of  the  magnitude  of  the  difference  of  the  a posteriori 
probabilities  of  the  transmitted  signals.  The  code  is  applied  to  systems  transmitting  sequences 
(words)  of  binary  digits  that  are  corrupted  by  the  addition  of  white  Gaussian  noise  and  then  de- 
tected by  correlation.  The  simplicity  of  the  calculations  described  in  Secs.C  and  E derives  from 
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the  fact  that  the  a posteriori  probability  p(x^/y)  that  xi  was  sent  if  y is  received  is  proportional 
(for  suitable  x.)  to  the  exponential  of  the  finite-time  correlation 


Under  the  assumptions  made  in  Sec.  B,  z.  is  normally  distributed,  and  a pair  of  signals  x^  and 
x^  can  be  chosen  such  that  their  correlations  are  statistically  independent. 

Unlike  the  Hamming  code,  the  Wagner  code  has  only  a probability  of  correcting 
single  errors.  However,  unless  the  signal-to-noise  ratio  of  the  correlator  difference  voltage 
is  less  than  unity  or  the  word  is  too  long,  this  probability  is  high.  The  Wagner  code  requires 
only  a single  check  digit,  the  burden  of  its  operation  being  placed  on  a memory  which  must  store 
the  values  of  the  a posteriori  probabilities  (or  their  differences)  for  the  duration  of  a word.  The 
Hamming  code,  on  the  other  hand,  requires  several  check  digits  (see  Sec.  A). 

If  the  words  are  transmitted  at  a constant  rate,  the  coded  words  (message  digits  and 
check  digits)  must  have  the  same  duration  as  the  uncoded  words  (message  digits  only);  therefore, 
the  digits  composing  the  word  must  be  appropriately  shortened.  This  increases  the  probability 
of  error  per  digit  and  diminishes  the  advantage  of  the  code.  Since  the  digits  of  a Wagner-coded 
word  are  longer  than  those  of  the  corresponding  Hamming-coded  word,  they  are  less  likely  to 
be  incorrectly  received.  Thus  it  is  natural  to  ask,  first,  whether  coding  is  worth  the  trouble  in 
a constant -data- rate  system,  and  second,  whether  the  Wagner  code  which  shortens  the  digits 
less  but  corrects  only  a certain  percentage  of  single  errors  is  better  than  the  Hamming  code 
which  shortens  the  digits  more  but  corrects  all  single  errors. 

These  questions  are  answered  m Sec.  D.  It  is  found  that  both  codes  are  valuable 
even  for  very  short  words,  but  that  the  Wagner  code  makes  considerably  fewer  errors,  depend- 
ing on  the  length  of  the  word  and  the  signal-to-noise  ratio  of  the  correlator  difference  voltage. 
(No  attempt  is  made  to  relate  channel  and  output  signal-to-noise  ratios.) 

The  successful  operation  of  the  Wagner  code  does  not  require  great  precision  of  the 
correlators  or  memory,  as  shown  in  Sec.  E. 

The  numerical  computations  reported  here  were  done  by  Mrs.  Elizabeth  Munro. 
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APPENDIX 

ON  THE  DISTRIBUTION  OF  FINITE-TIME  CORRELATIONS 


Consider  a transmitter  sending  one  of  two  signals  Xj(t)  and  x2(t)  of  duration  T and 
bandwidth  W.  Suppose  that  they  are  corrupted  by  the  addition  of  white  Gaussian  noise.  Thus, 
if  Xj(t)  is  sent,  the  received  signal  is 

y(t)  = Xj(t)  + n(t)  , (A-l) 


where  n(t)  is  a representative  noise  waveform. 


and 


x^t)  y(t)  dt 


x2(t)  y(t)  dt 


Let  Zj  and  z^  denote  the  finite-time  correlations 


(A-2) 


In  Sec.  B it  was  stated  without  proof  that  Zj  and  are  normally  distributed  random  variables. 

In  Sec.  C it  was  stated  without  proof  that  x (t)  and  x?(t)  ran  be  rhosen  so  that  z,  and  z,  are  sta- 

1 c.  1 z 

l istically  independent.  This  appendix  is  devoted  to  a justification  of  these  statements. 

In  order  to  prove  the  first  statement,  we  use  a well-known  theorem  of  Shannon8  that 

any  signal  of  bandwidth  W is  uniquely  determined  by  its  values  at  sample  points  spaced  l/2W 

seconds  apart.  Thus  x^t).  x2(t)  and  n(t)  each  can  be  written  in  the  form* 


whe  re 


2TW 

f(t)  = £ f(r/2W)  sine  (2Wt  -r) 

r=  1 


sine  x = sin  nx/trx 

The  ''unction  sine  x satisfies  the  orthonormality  property 

sine  (x  - r)  sine  (x  - s)  dx  = 6 
' rs 


( A - 3 ) 
(A-4) 


(A-S) 


Moreover,  if  the  spectrum  of  the  Gaussian  noise  n(t)  is  flat,  its  values  at  the  2TW  sample  points 
are  statistically  independent.8  Combining  Eqs.(A-l)  to  ( A -5 ),  we  see  that  Zj  and  z2  are  finite 
sums  of  statistically  independent  Gaussian  variables,  and  thus  are  themselves  Gaussian  random 
variables.  Moreover,  since  Zj  and  z£  are  derived  from  the  same  Gaussian  random  variables, 
namely,  the  sample  poim  values  of  n(t).  their  joint  distribution  is  two-dimensional  Gaussian.9 

To  prove  the  second  statement,  it  suffices  to  choose  Xj(t)  and  x2(t)  such  that  z^z"  = 
ZjZ2  (where  the  averages  are  over  the  noise  statistics),  since  linear  independence  implies  sta- 
tistical independence  for  two  variables  whose  joint  distribution  is  Gaussian.10  The  averages  of 


•It  w.H  be  seen  thot  f(t)  os  given  by  (A-3)  does  not  vanish  for  t < 0 and  t > T.  However,  for  lorge  TW,  f(t)  is  negligibly 
small  outside  the  finite  interval.  This  justifies  the  use  of  ideolized  signals  of  duration  T ond  bandwidth  W. 
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z.  and  z,  are 


1 = Z1  ‘ f xi-vdt=  f (Xj  + n)  Xj  dt 
J O o 

rT— ; rT  2 

= / (x.  + Xj  n)dt  - j Xj  dt  (since  x j n = x.  n 

J o ^ o 


(A-6) 


0). 


and 


°2  = Z2 


= x2  y dt  = C (x  j + n)  x^  dt  = C (x . x,  + x,  n)  dt 
-'o  J o 

rT  — 

/ Xj  xz  dt  (since  x2  n = x2  n = 0). 

U o 


(A-7) 


The  calculation  of  Zj  z2  is  more  complicated.  As  a first  step,  write  Zj  z2  as  the 
double  integral  


l1z2zJ'  xi  y dl  f *2  y'  dt'  = f f Xj  yx2'  y'  dt  dt' 


(A-8) 


where  Xj  and  y are  written  for  argument  t [as  in  Eqs.(A-6)  and  (A-7)],  and  x2'  and  y1  for  argu- 
ment t'.  Then,  substituting  y = Xj  + n,  y*  = x j'  + n1,  and  expanding,  we  obtain 

*T  rT  — — — 

(Xj  Xj1  x2'  + nXj  xj  x2'  tn'Xj  x2'  + nn'  x t x.,')  dt  dt’ 


X X 


T r t 


l'  XjX-,'  dt  dt'  (using  n = n'  = 0)  (A-9) 


- Z1  Z2  + 


f J Rn<1  ~ *')  xi<t)  x2(t')  dt  dt’ 


where  Rn  is  the  autocorrelation  function  of  the  noise  n.  Since  Xj  and  x2  are  negligibly  small 
outside  the  interval  0 < t < T,  the  last  integral  in  Eq.(A-9)  can  be  approximated  by 


where 


/•CO  pCO  f*  00 

I J R (t  - f)  Xj(t)  x2(f)  dt  dt'  = I R (t)R  (T)dt 
J -00^  -00  *00 

/oo 

Xj(t  + T)  x2(t)  dt  . 

OO 


(A- 10) 


( A-  1 1 ) 


We  can  choose  x(  and  x2  such  that  Rj2(t)  is  very  small  where  Rn(T)  is  significant,  that  is,  for 
t<  1/W.  Indeed,  if  Xj  and  x2  have  nonoverlapping  energy  spectra  within  the  band  W,  R12(T)  2 0. 
Thus,  for  suitably  chosen  Xj  and  x2>  we  may  neglect  the  integral  Eq.(A-lO)  , so  that  Eq.(A-9) 
becomes 

=1*2  • (A- 12) 


•This  meons  thot  x,  ond  x2  are  approximately  orthogonol  ond  remoin  so  when  time-displaced  with  respect  to  eoch  other 
by  an  amount  ^ the  correlation  time  of  the  noise. 
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Equation  (A- 12)  states  that  Zj  and  z.,  are  (approximately)  linearly  independent  and  consequently 
(approximately)  statistically  independent. 

The  results  just  derived  are  equally  valid  if  the  band  W starts  at  WQ  = nW,  with  n 

an  integer.  They  are  approximately  va!H  in  any  case  if  W » W,  which  is  usually  the  case  in 

11  ° 

communication  problems. 
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